Pandas SeabornΒΆ
In this notebook we'll look at interfacing between the composability and ability to generate complex visualizations that HoloViews provides, the power of pandas library dataframes for manipulating tabular data, and the great looking statistical plots and analyses provided by the Seaborn library.
We also explore how a pandas DFrame can be wrapped in a general purpose Element type, which can either be used to convert the data into other standard Element types or be visualized directly using a wide array of Seaborn-based plotting options, including:
- regression plots
- correlation plots
- box plots
- autocorrelation plots
- scatter matrices
- histograms
- scatter or line plots
This tutorial assumes you're already familiar with some of the core concepts of HoloViews, which are explained in the other Tutorials.
This tutorial requires NumPy, Pandas, and Seaborn to be installed and imported:
import itertools
import numpy as np
import pandas as pd
import seaborn as sb
np.random.seed(9221999)
import holoviews
from holoviews import *
We can now select static and animation backends:
%load_ext holoviews.ipython
%output holomap='widgets' fig='svg'
Visualizing Distributions of Data ¶
If import seaborn succeeds, HoloViews will provide a number of additional Element types, including Distribution, Bivariate, TimeSeries, Regression, and DFrame (a Seaborn-visualizable version of the DFrame Element class provided when only pandas is available).
We'll start by generating a number of Distribution Elements containing normal distributions with different means and standard deviations and overlaying them. Using the %%opts magic you can specify specific plot and style options as usual; here we deactivate the default histogram and shade the kernel density estimate:
%%opts Distribution (hist=False kde_kws=dict(shade=True))
d1 = 25 * np.random.randn(500) + 450
d2 = 45 * np.random.randn(500) + 540
d3 = 55 * np.random.randn(500) + 590
Distribution(d1, label='Blue') *\
Distribution(d2, label='Red') *\
Distribution(d3, label='Yellow')
Thanks to Seaborn you can choose to plot your distribution as histograms, kernel density estimates, or rug plots:
%%opts Distribution (rug=True kde_kws={'color':'indianred','linestyle':'--'})
Distribution(np.random.randn(10), key_dimensions=['Activity'])
We can also visualize the same data with Bivariate distributions:
%%opts Bivariate.A (shade=True cmap='Blues') Bivariate.B (shade=True cmap='Reds') Bivariate.C (shade=True cmap='Greens')
Bivariate(np.array([d1, d2]).T, group='A') +\
Bivariate(np.array([d1, d3]).T, group='B') +\
Bivariate(np.array([d2, d3]).T, group='C')
This plot type also has the option of enabling a joint plot with marginal distribution along each axis, and the kind option lets you control whether to visualize the distribution as a scatter, reg, resid, kde or hex plot:
%%opts Bivariate [joint=True] (kind='kde' cmap='Blues')
Bivariate(np.array([d1, d2]).T, group='A')
Bivariate plots also support overlaying and animations, so let's generate some two dimensional normally distributed data with varying mean and standard deviation.
Working with TimeSeries data¶
Next let's take a look at the TimeSeries View type, which allows you to visualize statistical time-series data. TimeSeries data can take the form of a number of observations of some dependent variable at multiple timepoints. By controlling the plot and style option the data can be visualized in a number of ways, including confidence intervals, error bars, traces or scatter points.
Let's begin by defining a function to generate sine wave time courses with varying phase and noise levels.
def sine_wave(n_x, obs_err_sd=1.5, tp_err_sd=.3, phase=0):
x = np.linspace(0+phase, (n_x - 1) / 2+phase, n_x)
y = np.sin(x) + np.random.normal(0, obs_err_sd) + np.random.normal(0, tp_err_sd, n_x)
return y
Now we can create HoloMaps of sine and cosine curves with varying levels of observational and independent error.
sine_stack = holoviews.HoloMap(key_dimensions=['Observation error','Random error'])
cos_stack = holoviews.HoloMap(key_dimensions=['Observation error', 'Random error'])
for oe, te in itertools.product(np.linspace(0.5,2,4), np.linspace(0.5,2,4)):
sines = np.array([sine_wave(31, oe, te) for _ in range(20)])
sine_stack[(oe, te)] = TimeSeries(sines, label='Sine', group='Activity',
key_dimensions=['Time', 'Observation'])
cosines = np.array([sine_wave(31, oe, te, phase=np.pi) for _ in range(20)])
cos_stack[(oe, te)] = TimeSeries(cosines, group='Activity',label='Cosine',
key_dimensions=['Time', 'Observation'])
First let's visualize the sine stack with a confidence interval:
%%opts TimeSeries [apply_databounds=True] (ci=95 color='indianred')
sine_stack
And the cosine stack with error bars:
%%opts TimeSeries (err_style='ci_bars')
cos_stack.last
Since the %%opts cell magic has applied the style to each object individually, we can now overlay the two with different visualization styles in the same plot:
cos_stack.last * sine_stack.last
Let's apply the databounds across the HoloMap again and visualize all the observations as unit points:
%%opts TimeSeries (err_style='unit_points')
sine_stack * cos_stack
Working with pandas DataFrames¶
In order to make this a little more interesting, we can use some of the real-world datasets provid3ed with the Seaborn library. The holoviews DFrame object can be used to wrap the Seaborn-generated pandas dataframes like this:
iris = DFrame(sb.load_dataset("iris"))
tips = DFrame(sb.load_dataset("tips"))
titanic = DFrame(sb.load_dataset("titanic"))
By default the DFrame simply inherits the column names of the data frames and converts them into Dimensions. This works very well as a default, but if you wish to override it, you can either supply an explicit list of key_dimensions to the DFrame object or a dimensions dictionary, which maps from the column name to the appropriate Dimension object. In this case, we define a Month Dimension, which defines the ordering of months:
flights_data = sb.load_dataset('flights')
dimensions = {'month': Dimension('Month', values=list(flights_data.month[0:12])),
'passengers': Dimension('Passengers', type=int),
'year': Dimension('Year', type=int)}
flights = DFrame(flights_data, dimensions=dimensions)
%output fig='png' dpi=100 size=150
Flight passenger data¶
Now we can easily use the conversion methods on the DFrame object to create HoloViews Elements, e.g. a Seaborn-based TimeSeries Element and a HoloViews standard HeatMap:
%%opts TimeSeries (err_style='unit_traces' err_palette='husl') HeatMap [xrotation=30]
flights.timeseries(['Year', 'Month'], 'Passengers', label='Airline', group='Passengers') +\
flights.heatmap(['Year', 'Month'], 'Passengers', label='Airline', group='Passengers')
Tipping data ¶
A simple regression can easily be visualized using the Regression Element type. However, here we'll also split out smoker and sex as Dimensions, overlaying the former and laying out the latter, so that we can compare tipping between smokers and non-smokers, separately for males and females.
%%opts Regression [apply_databounds=True]
tips.regression('total_bill', 'tip', mdims=['smoker','sex'],
extents=(0, 0, 50, 10), reduce_fn=np.mean).overlay('smoker').layout('sex')
When you're dealing with higher dimensional data you can also work with pandas dataframes directly by displaying the DFrame Element directly. This allows you to perform all the standard HoloViews operations on more complex Seaborn and pandas plot types, as explained in the following sections.
Iris Data ¶
Let's visualize the relationship between sepal length and width in the Iris flower dataset. Here we can make use of some of the inbuilt Seaborn plot types, a pairplot which can plot each variable in a dataset against each other variable. We can customize this plot further by passing arguments via the style options, to define what plot types the pairplot will use and define the dimension to which we will apply the hue option.
%%opts DFrame (diag_kind='kde' kind='reg' hue='species')
iris.clone(label="Iris Data", plot_type='pairplot')
When working with a DFrame object directly, you can select particular columns of your DFrame to visualize by supplying x and y parameters corresponding to the Dimensions or columns you want visualize. Here we'll visualize the sepal_width and sepal_length by species as a box plot and violin plot, respectively.
%%opts DFrame [show_grid=False]
iris.clone(x='species', y='sepal_width', plot_type='boxplot') + iris.clone(x='species', y='sepal_length', plot_type='violinplot')
Titanic passenger data ¶
The Titanic passenger data is a truly large dataset, so we can make use of some of the more advanced features of Seaborn and pandas. Above we saw the usage of a pairgrid, which allows you to quickly compare each variable in your dataset. HoloViews also support Seaborn based FacetGrids. The FacetGrid specification is simply passed via the style options, where the map keyword should be supplied as a tuple of the plotting function to use and the Dimensions to place on the x axis and y axis. You may also specify the Dimensions to lay out along the rows and columns of the plot, and the hue groups:
%%opts DFrame (map=('barplot', 'alive', 'age') col='class' row='sex' hue='pclass' aspect=1.0)
titanic.clone(plot_type='facetgrid')
FacetGrids support most Seaborn and matplotlib plot types:
%%opts DFrame (map=('regplot', 'age', 'fare') col='class' hue='class')
titanic.clone(plot_type='facetgrid')
Finally, we can summarize our data using a correlation plot and split out Dimensions using the .holomap method, which groups by the specified dimension, giving you a frame for each value along that Dimension. Here we group by the survived Dimension (with 1 if the passenger survived and 0 otherwise), which thus provides a widget to allow us to compare those two values.
%%output holomap='widgets' size=200
titanic.clone(titanic.data.dropna(), plot_type='corrplot').holomap(['survived'])
As you can see, the Seaborn plot types and pandas interface provide substantial additional capabilities to HoloViews, while HoloViews allows simple animation, combinations of plots, and visualization across parameter spaces. Note that the DFrame Element is still available even if Seaborn is not installed, but it will use the standard HoloViews visualizations rather than Seaborn in that case.